1. Field of the Invention
This invention relates to a video signal coding method and a video signal encoder that can highly efficiently and effectively encode digital signals at the transmission side. More particularly, the present invention relates to a video signal coding method and a video signal encoder adapted to control a coding operation conducted at a variable bit rate on a single path when coding moving picture signals.
2. Description of the Related Art
Since digital video signals involve a large amount of data, the use of a high efficiency coding means that can encode a digital video signal at a high compression ratio is indispensable particularly when recording signals on a small recording medium having a small storage capacity. High efficiency coding methods utilizing the correlation of video signals have been proposed as solutions for meeting the need. One of such solutions is the use of the MPEG system. The MPEG (Moving Picture Inage Coding Experts Group) system was proposed as standard system as a result of the discussion in ISO-IEC/JTC1/SC2/WG11. It is a hybrid system realized by combining motion compensation predictive coding and discrete cosine transform (DCT). With the MPEG system, firstly the redundancy is eliminated on the time base by determining the inter-frame differences of video signals and subsequently on the space base by means of discrete cosine transform to efficiently encode video signals.
Generally, video signals do not remain invariable and the amount of information of each picture changes with time. Thus, it is known that a high image quality can be realized by means of variable bit rate coding rather than by means of constant bit rate coding if a same amount of codes is used.
For example, 2-pass type variable bit rate coding is generally used for video signals to be recorded on a DVD-video. The 2-pass type involves two coding operations including one for determining the amount of codes and one to be conducted by variably controlling the bit rate on the basis of the determined amount of codes. While this technique provides the advantage of effectively exploiting the total amount of coding bits that can be used, it is accompanied by the disadvantage that a time period more than twice of the duration of the moving picture sequence is required for the processing operation so that this technique is not suited to real time processing operations.
The specifications and the drawings of Japanese Patent Applications Laid-Open Nos. 7-311418 and 9-113141 disclose a 1-pass type variable bit rate coding method in an attempt for reducing the processing time.
FIG. 1 of the accompanying drawings schematically illustrates the configuration of an encoder for moving pictures realized by applying the known 1-pass type variable bit rate coding method and FIG. 2 sows a flow chart of the processing operation using the 1-pass type variable bit rate coding method.
Referring to FIG. 1 showing a block diagram of a known moving picture encoder, the input moving image signal applied to input terminal 200 is sent to coding difficulty level computation circuit 201 and delay circuit 203. Then, the output of the coding difficulty level computation circuit 201 is sent to amount of allocated bits computation circuit 202 that is adapted to compute the amount of allocated bits per unit time and the output of the amount of allocated bits computation circuit 202 is sent to moving image coding circuit 204. The moving image coding circuit 204 encodes the output signal of the delay circuit 203 as a function of the amount of allocated bits as output from the amount of allocated bits computation circuit 202 and outputs the coded signal as coded bit stream from terminal 205.
Now, the operation of the moving picture encoder of FIG. 1 will be described by referring to the flow chart of FIG. 2.
Referring to FIG. 2, in Step S401, the moving picture signal fed to the terminal 200 is input to the coding difficulty level computation circuit 201 to determine the coding difficulty level d of the input image per unit time, which unit time may typically be 0.5 seconds. The computing operation of the coding difficulty level computation circuit 201 is typically performed by encoding the input moving picture, fixing the quantization step, and computing the amount of generated codes in a predetermined period of time.
Then, in Step S402, the amount of allocated bits computation circuit 202 computationally determines the amount of allocated bits b relative to the coding difficulty level d obtained by the coding difficulty level computation circuit 201. For this operation, the relationship between the coding difficulty level d and the amount of allocated bits b observable when a reference moving image sequence is subjected to variable bit rate coding at a predetermined average bit rate is obtained in advance for each unit time. The amount of allocated bits per unit time for the reference moving image sequence is held to less than the storage capacity of the recording medium to be used for signal recording. FIG. 3 shows a graph that illustrates the relationship between the coding difficulty level d and the amount of allocated bits b.
Referring to FIG. 3, the horizontal axis of the graph represents the coding difficulty level d and the vertical axis of the graph represents the probability of appearance h (d) of coding difficulty level d in the reference moving image sequence. The amount of allocated bits for any given coding difficulty level is computed on the basis of function b (d). This function is empirically obtained on a trial and error basis by conducting a number of experiments of encoding many moving image sequences (e.g., movies) at a predetermined average bit rate and evaluating the image quality of each sequence. In other words, it represents a generic relationship applicable to any moving image sequences that may be observable in this world. The specification and the drawings of Japanese Patent Application Laid-Open No. 7-311418 discloses a method of obtaining such a function. Thus, the amount of allocated bits computation circuit 202 determines the amount of allocated bits b for the coding difficulty level d per unit time of the image that is input through the terminal 200 on the basis of the relationship as illustrated in FIG. 3.
Since the processing operation of the coding difficulty level computation circuit 201 and the amount of allocated bits computation circuit 202 on the input image of a unit time length completes within the unit time, the delay circuit 203 is provided in the 1-pass type moving picture encoder to delay the input of the image signal to the moving image coding circuit 204 by a unit time.
Then, in Step S403, the moving image coding circuit 204 encodes the input moving image of each unit time so as to show the amount of allocated bits as given by the amount of allocated bits computation circuit 202 for the moving image. In other words, the moving image coding circuit 204 encodes the input moving image of each unit time, using the quantization step size that is defined on the basis of the amount of allocated bits.
With such a 1-pass type method, it is possible to perform an operation of variable bit rate coding on the input video signal with an optimal amount of allocated bits that corresponds to the coding difficulty level of the signal substantially on a real time basis.
Meanwhile, although the relationship of FIG. 3 is applicable to almost all moving image sequences, it is not to certain special sequences where the total amount of bits generated by the moving image coding circuit 204 exceeds the total amount of available bits so that the moving image sequence cannot be recorded on the intended recording medium.
The conventional 2-pass type method cannot be used for encoding a moving image sequence having a predetermined time length at a variable bit rate on a real time basis and recording it in a recording medium having a certain level of storage capacity. On the other hand, while the conventional 1-pass type method can encode almost all moving image sequences at a variable bit rate on a real time basis and store a sequence having a predetermined time length in a recording medium having a certain level of storage capacity, it is not applicable to certain special sequences where the total amount of coded bits exceeds the total amount of available bits so that the sequence having a predetermined time length cannot be recorded on a recording medium.
In view of this problem, there has been proposed a technique of guaranteeing that a signal having a predetermined time length can be recorded on a recording medium having a certain level of storage capacity by controlling the amount of actually allocated bits in such a way that the sum of the bits generated when a signal having a time length good for recording on a recording medium is encoded is less than the amount of bits that can be used for recording the signal on the recording medium.
More specifically, assuming that the input signal of each unit time is encoded with a predetermined amount of allocated bits b_av, the sum of the amounts of allocated bits B_av down to the current time and the sum of the amounts of actually generated coding bits B_gen down to the current time are compared and if the value (B_av−B_gen) is positive, it is permitted to give an amount of allocated bits more than b_av for the guarantee.
The predetermined amount of allocated bits b_av per unit time is determined by the formula below;b—av=T_unit time×B—V/T—SEQ where                B_V: the amount of bits available to recording the moving image in the signal recording medium,        T_SEQ: the time length of a moving image sequence that can be recorded in the signal recording medium, and        T_unit time: unit time length.        
The input signal is pre-filtered on the basis of the information obtained when shifting the reference value of the amount of allocated coding bits to the actual value of the amount of allocated coding bits and then the obtained signal is coded to make the degradation of the signal due to the coding less conspicuous. More specifically, when the actual value of the amount of allocated coding bits is made smaller than the reference value of the amount of allocated coding bits, the degradation of the signal due to the coding less conspicuous by causing the input image to pass through a low pass filter.
When the operation of coding a signal and recording the coded signal on a recording medium is divided into segments of operation that are to be carried out sequentially, the above (B_av−B_gen) or a corresponding value is stored in the recording medium. Thus, when the signal is stored in a vacant storage area of the recording medium, the above (B_av−B_gen) or the corresponding value is read out from recording medium and the amount of bits allocated per unit time to the signal is computed on the basis of the read out value so that the storage capacity of the recording medium can be utilized effectively and efficiently.
When the input signal is that of a moving image, the coding difficulty level is determined on the basis of the information on the image characteristics of the input image for every predetermined period of time and the signal is coded on the basis of the amount of allocated coding bits that reflects the visual characteristics of human being, using the information on the image characteristics.
FIG. 4 shows a block diagram of an encoder for moving pictures realized by applying a 1-pass type variable bit rate coding method that guarantees that a signal having a predetermined length on a recording medium having a certain level of storage capacity.
Referring to FIG. 4, the moving image signal S1 input to terminal 101 is sent to a motion vector computation circuit 102. The motion vector computation circuit 102 outputs information on the motion vector (the motion vector and the prediction error) of the input moving image. The moving image signal and the motion vector information are then input to a coding difficulty level computation circuit 103. The coding difficulty level computation circuit 103 computes the coding difficulty level d per unit time of the input moving image S1. The obtained coding difficulty level d is then input to the amount of allocated bits computation circuit 104 that computes the amount of allocated bits per unit time. More specifically, the amount of allocated bits computation circuit 104 computes reference value b of the amount of bits allocated per unit time for coding the input image. The reference value b of the amount of allocated bits is then input to a controller 105. The controller 105 shifts the reference value b of the amount of allocated bits to the actual value b_x of the amount of allocated bits and outputs it. The controller 105 also outputs information S4 on the processing operation of a pre-filter 107. A Delay circuit 106 delays the input of the input moving image signal S1 having the length of the unit time to the pre-filter 107 by the unit time, or until the processing operation of the amount of allocated bits computation circuit 104 and that of the controller 105 are completed. The input moving image signal S2 delayed by the delay circuit 106 is processed by the pre-filter 107 according to the processing information S4 and the pre-filter 107 outputs the processed signal S3. The pre-filter 107 is adapted to change its filtering characteristics according to the processing information S4 so as to prevent the possible degradation of the input moving image signal at the time of coding by causing the signal to pass through a low pass filter when the input moving image is a complex one and requires a large amount of bits allocated to it per unit time. Moving image coding circuit 108 encodes the processed image signal S3 so as to make it show the amount of allocated bits b_x per unit time. The moving image coding circuit 108 then outputs coded bit stream S5 and the amount of generated bits b_gen per unit time, of which the coded bit stream S5 is output from terminal 109.
With such a 1-path type method, the actual value of the amount of allocated bits can be controlled in such a way that input moving image can be coded at a variable bit rate by using an optimal amount of allocated bits according to the coding difficulty level of the signal on a real time bases and the total sum of the amounts of generated bits obtained when coding the signal having a time length that allows the signal to be recorded on the recording medium is less than the amount of bits available for recording the signal on the recording medium.
However, the above described method for determining the amount of allocated bits according to the coding difficulty level is accompanied by a problem that, when the input moving image signal shows a high coding difficulty level concentratively in the initial stages, (B_av−B_gen) would not show a positive value and hence it is not possible to improve the image quality because a sufficient amount of bits cannot be allocated per unit time for the part of the image signal showing a high initial coding difficulty level if a latter part of the image signal shows a low coding difficulty level.
Additionally, there is another problem that accompanies the above method. When (B_av−B_gen) becomes positive so that it is possible to allocate an amount of bits per unit time that is greater than the ordinary level and the input moving image signal continuously shows a high coding difficulty level so that an amount of bits per unit time that is greater than the ordinary level is actually allocated but subsequently (B_av−B_gen) suddenly becomes negative on the way, the amount of allocated bits per unit time falls abruptly.
On the other hand, when the input moving image signal continuously shows a low coding difficulty level, there also arises a problem that the allocation of an amount of bits per unit time that is smaller than the ordinary level goes on to degrade the quality of the recorded image to a level lower than the image quality that is obtained when the image is coded at a fixed coding rate, although (B_av−B_gen) is positive and the amount of bits per that is allocated per unit time is greater than the ordinary level.
Additionally, there is also a problem that, since the maximum amount of allocated bits and the minimum amount of allocated bits are not defined in actual operations, there can occur parts where the amount of allocated bits is extremely large and parts where the amount of allocated bits is extremely small after the coding operation and hence it is not possible to allocate bits to the input image in a well coordinated manner.
Finally, although the image quality of an image can appear to be degraded in some parts to the visual sense of human being while it does not in some other parts if bits are allocated at a same rate, no satisfactory measures have been taken to such a problem.